Two modifications of Levenberg-Marquardt's method for fast batch neural network training
نویسندگان
چکیده
The problems of artificial neural networks learning and their parallelisation are taken up in this article. The article shows comparison of the Levenberg-Marquardt’s method (LMM) and its two modifications JWM (method with Jacobian matrices formed in each step) and BKM (Jacobian calculations only in the first step) for training artificial neural networks. These algorithms have the following properties: 1) simpler calculations; 2) they are partly parallelized. The experiments proved their efficiency. Experimental results demonstrate that neural network for training by them needs a similar number of epochs as the LMM and lesser time for training.
منابع مشابه
An Algorithm for Fast Convergence in Training Neural Networks
In this work, two modifications on Levenberg-Marquardt algorithm for feedforward neural networks are studied. One modification is made on performance index, while the other one is on calculating gradient information. The modified algorithm gives a better convergence rate compared to the standard Levenberg-Marquard (LM) method and is less computationally intensive and requires less memory. The p...
متن کاملPrediction of Groundwater Levels Using Different Artificial Neural Network Architectures and Algorithms
Performance of four types of functionally different artificial neural network (ANN) models, namely Feed forward neural network, Elman type recurrent neural network, Input delay neural network and Radial basis function network and fourteen types of algorithms, namely Batch gradient descent (traingd), Batch gradient descent with momentum (traingdm), Adaptive learning rate (traingda), Adaptive lea...
متن کاملNumerical and Neural Network Modeling and control of an Aircraft Propeller
In this paper, parametric and numerical model of the DC motor, connected to aircraft propellers are extracted. This model is required for controlling trust and velocity of the propellers, and consequently, an aircraft. As a result, both of torque and speed of the propeller can be controlled simultaneously which increases the kinematic and kinetic performance of the aircraft. Parametric model of...
متن کاملImplementing the Levenberg-marquardt Algorithm On-line: a Sliding Window Approach with Early Stopping
The Levenberg-Marquardt algorithm is considered as the most effective one for training Artificial Neural Networks but its computational complexity and the difficulty to compute the trust region have made it very difficult to develop a true iterative version to use in on-line training. The algorithm is frequently used for off-line training in batch versions although some attempts have been made ...
متن کاملTraining recurrent network with block-diagonal approximated Levenberg-Marquardt algorithm
In this paper, we propose the block-diagonal matrix to approximate the Hessian matrix in the Levenberg Mar-quardt method in the training of neural networks. Two weight updating strategies, namely asynchronous and synchronous updating methods were investigated. Asyn-chronous method updates weights of one block at a time while synchronous method updates all weights at the same time. Variations of...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
- Annales UMCS, Informatica
دوره 2 شماره
صفحات -
تاریخ انتشار 2004